Item Response Theory Model Assumptions Research Articles

Detecting uniform differential item functioning for continuous response computerized adaptive testing

Evaluating items for potential differential item functioning (DIF) is an essential step to ensuring measurement fairness. In this article, we focus on a specific scenario, namely, the continuous response, severely sparse, computerized adaptive testing (CAT). Continuous responses items are growingly used in performance-based tasks because they tend to generate more information than traditional dichotomous items. Severe sparsity arises when many items are automatically generated via machine learning algorithms. We propose two uniform DIF detection methods in this scenario. The first is a modified version of the CAT-SIBTEST, a non-parametric method that does not depend on any specific item response theory model assumptions. The second is a regularization method, a parametric, model-based approach. Simulation studies show that both methods are effective in correctly identifying items with uniform DIF. A real data analysis is provided in the end to illustrate the utility and potential caveats of the two methods.

Lower Quadrant Edema Patient-Reported Outcome Measure Is Reliable, Valid, and Efficient for Patients with Lymphatic and Venous Disorders.

The main aims of this study were: (1) to create a patient-reported outcome measure (PROM) item bank for measuring the impact of lower quadrant edema (LQE) on physical function using item response theory and (2) to assess reliability, validity, and administration efficiency of LQE PROM scores based on computerized adaptive test (CAT) and the reliability of a 10-item short form (SF). This retrospective study included data from patients treated in outpatient rehabilitation clinics for lower quadrant edema who responded to all 30 candidate items at intake. Item response theory model assumptions of unidimensionality, local item independence, item fit, and presence of differential item functioning were evaluated. LQE-CAT-generated scores were assessed for reliability, validity, and administration efficiency. LQE-SF-generated scores were assessed for reliability. The total cohort included 4894 patients (mean [SD] age = 65 [14] y; range = 14-89 y). A set of 20 items was selected for the item bank based on support for its unidimensionality and fit to the item response theory model, with reliability estimates greater than 0.92 for CAT and SF administration modes. No items demonstrated differential item functioning with respect to tested variables. After controlling for scores at intake, scores discriminated among multiple patient groups in clinically logical ways with better outcomes observed for patients who were younger with less chronic symptoms and fewer comorbidities. Scores were responsive to change but the effect size was small (0.4). There were negligible floor and ceiling effects. CAT administration of the item bank required an average of 6.1 items (median = 5). Scores correlated highly with full-bank scores (Pearson correlation coefficient = 0.98). Scores on the LQE PROM were reliable, valid, and efficient for assessing perceived physical function of patients with lower quadrant edema. The LQE CAT and SF are suitable for research and routine clinical care. Reasons for the small effect size for change scores should be studied. The newly developed LQE PROM was reliable and valid and offers efficient administration modes for assessing perceived physical function of patients with LQE, both for research and routine clinical care in busy outpatient rehabilitation settings. As an item response theory-based measure, the LQE PROM allows administration of condition-specific functional questions with low response burden for patients. The 10-item LQE-SF offers a feasible alternative administration mode when CAT administration is not available. This study supports a transition to PROMs that are based on modern measurement approaches to achieve the combined benefits of high accuracy and efficiency.

The Stroke Upper and Lower Extremity Physical Function Measures Were Supported for Score Reliability, Validity, and Administration Efficiency for Patients Poststroke.

The aims of this study were to (1) evaluate the suitability of newly developed items for calibration into 2 item banks for stroke upper extremity (SUE) and stroke lower extremity (SLE) physical function patient-reported outcome measures (PROMs) and (2) assess score reliability and validity and PROM administration efficiency based on computerized adaptive testing (CAT). A retrospective longitudinal study involving patients poststroke who were treated in outpatient rehabilitation clinics responded to 28 and 25 region-specific candidate items addressing tasks related to upper or lower extremity physical function, respectively. Item response theory model assumptions of unidimensionality, local independence, item fit, and presence of differential item functioning were evaluated. CAT-generated scores were assessed for reliability, validity, and administration efficiency, and 10-item short forms were assessed for reliability. Cohorts consisted of 2017 patients with stroke involving the upper extremity and 2107 patients with stroke involving the lower extremity (mean age [SD]: SUE = 62 [14] and SLE = 63 [14]; range = 14-89). Two solutions (SUE: 28-item; SLE: 24-item) supported unidimensionality and fit to the item response theory model, with reliability estimates >0.93 for all administration modes. No items demonstrated differential item functioning. Scores discriminated among multiple patient groups in clinically logical ways, with better outcomes observed for patients who were younger, were male, had less chronicity, and had fewer comorbidities. The SUE and SLE, respectively, had 1% and 0.3% floor effects and 4.3% and 1.1% ceiling effects. Change score effect sizes were 0.5 (SUE) and 0.6 (SLE). Simulated CAT scores required an average of 6 (SUE) and 5.6 (SLE) items (median = 5). The stroke upper extremity and stroke lower extremity PROM scores were reliable, valid, and efficient and had moderate change effect sizes for assessing physical function as perceived by patients poststroke with upper and lower extremity impairments. Scores had negligible floor and acceptable ceiling effects. Based on these results, the stroke PROMs are suitable for research and routine clinical practice. As item response theory-based measures, these PROMs support clinical practice guideline recommendations for the use of outcome measures in neurologic physical therapy and the administration of condition-specific functional questions with low response burden for patients. The 10-item short forms offer a feasible alternative administration mode when CAT administration is not available.

Reliability, Validity, and Efficiency of an Item Response Theory-Based Balance Confidence Patient-Reported Outcome Measure.

The aims of this study were to calibrate the original 16 items from the Activities-specific Balance Confidence (ABC) Scale to create an item response theory (IRT)-based item bank and scoring metric of balance confidence (BC) and to assess psychometric properties of a computerized adaptive test (BC-CAT) and 6-item short-form (BC-SF) administration modes. This retrospective study included data from patients who were treated in outpatient rehabilitation clinics and assessed for balance impairments by responding to the full ABC Scale at intake. IRT model assumptions of unidimensionality, local item independence, item fit, and presence of differential item functioning (DIF) were evaluated. BC-CAT-generated scores were assessed for reliability, validity, and administration efficiency, and the newly developed BC-SF was assessed for reliability. Total cohort included 20,354 patients (mean age [SD] = 66 [16] y; range = 14-89). All 16 items were retained in the final item bank based on support for unidimensionality and fit to the IRT model. No items demonstrated DIF. Reliability estimates were 0.95, 0.96, and 0.98 for the BC-SF, BC-CAT, and the full item bank, respectively. Scores discriminated among patient groups in clinically logical ways. After controlling for scores at intake, better outcomes were achieved for patients who were younger, had more acute symptoms, exercised more, and had fewer comorbidities. Scores were responsive to change with a moderate effect size, with negligible floor and ceiling effects. CAT scores were generated using an average of 4.7 items (median = 4) and correlated highly with full-bank scores (Pearson correlation coefficient = 0.99). The IRT-based BC patient-reported outcome measure (PROM) was reliable, valid, moderately responsive to change, and efficient, with excellent score coverage. The measure is suitable for research and routine clinical administration using the BC-CAT or BC-SF administration modes. The full ABC Scale can be administered for increased clinical content when appropriate. The newly developed BC patient-reported outcome measure (PROM) was reliable and valid for assessing perceived balance confidence. In addition, the BC PROM has efficient administration modes with low patient response burden, which enhances feasibility and use during routine clinical practice in busy rehabilitation settings. This study supports a transition to PROMs that are based on modern measurement approaches to achieve the combined benefits of high accuracy and efficiency.

New Item-response Theory-based Dizziness Impact Measures Were Reliable, Valid, and Efficient

<h3>Research Objectives</h3> Calibrate items from the Dizziness Handicap Inventory (DHI) patient-reported outcome measure (PROM) to create an item response theory (IRT)-based item bank and assess its reliability, validity, and efficiency using computerized adaptive test (CAT) and short form (SF) administration modes. <h3>Design</h3> A retrospective cohort including data from patients assessed for dizziness. <h3>Setting</h3> Outpatient rehabilitation. <h3>Participants</h3> Total cohort consisted of 28,815 adult patients [69% female; mean age (SD) 60 (18)] treated in 976 clinics from 49 states (USA) for either vertigo, brain injury, or neck impairments who responded to all 25 DHI items at intake. <h3>Interventions</h3> Not applicable. <h3>Main Outcome Measures</h3> IRT model assumptions of local independence, unidimensionality, item fit, and absence of differential item functioning (DIF) were evaluated. CAT and SF T-scores (mean=50, SD=10) were assessed for reliability, validity, and administration efficiency. Score known-groups validity was assessed for the ability to discriminate among patient groups by demonstrating expected differing levels of dizziness impact. Floor and ceiling effects were assessed, and responsiveness was evaluated using an effect-size statistic. <h3>Results</h3> Three unidimensional item sets were identified, creating 3 distinct PROMs to assess functional (DF; 13 items), positional (DP; 4 items), and emotional (DE; 6 items) status (higher score=lower impact). Each item fit its IRT model. Reliability estimates were 0.72 to 0.92. Except for DIF by age group for DF and DE, no items were flagged for DIF, with negligible DIF impact. Scores discriminated between patient groups in clinically expected ways. Floor and ceiling effects were acceptable (< 15%) excepting a floor effect for the positional PROM (21.4%). Effect sizes were large (>0.8). A CAT was developed for DF, administering an average of 8.3 items (median=9, range 5 to 10). <h3>Conclusions</h3> The 3 dizziness PROMs demonstrated moderate to high reliability, were valid, highly responsive to change; thus, they are suitable for routine clinical administration. <h3>Author(s) Disclosures</h3> Dr. Kallen consults for Net Health Systems, Inc., that owns the Focus on Therapeutic Outcomes (FOTO) Patient Outcomes system that gathers and manages the data analyzed. The authors have no other conflicts to declare.

Use of computerised adaptive testing to reduce the number of items in patient-reported hip and knee outcome scores: an analysis of the NHS England National Patient-Reported Outcome Measures programme

ObjectiveOver 160 000 participants per year complete the 12-item Oxford Hip and Knee Scores (OHS/OKS) as part of the NHS England Patient-Reported Outcome Measures (PROMs) programme. We used a modern...

Towards standardization of fatigue measurement: Psychometric properties and reference values of the PROMIS Fatigue item bank in the Dutch general population

Background There is little consensus on how to measure fatigue. Objectives To standardize the measurement of fatigue across populations, we aimed to assess the psychometric properties of the PROMIS Fatigue item bank in the Dutch general population and obtain reference values. Methods A sample of 1006 people participating in an internet panel completed the full v1.0 PROMIS Fatigue item bank (95 items). Structural validity (item response theory (IRT) assumptions and IRT model fit), measurement invariance/cross-cultural validity (absence of differential items functioning (DIF) for demographic variables and language, compared to data from US participants in PROMIS wave 1), and (internal) reliability (percentage of respondents with reliable estimates) were assessed. Results The IRT model assumptions were considered met (ECV 0.86, Omega-H 0.92), all items fitted the IRT model, no items showed DIF for demographic variables and seven for language, but with negligible impact on T-scores. Reliable fatigue T-scores were found for 98.3%, 69.8–82.6%, and 96.5% of the respondents with the full item bank, the standard short forms, and a simulated computerized adaptive test (CAT), respectively. The CAT administered on average only five items. A T-score of 49.1 represented the average score of the Dutch general population, T-scores &lt;55 are considered within normal limits, T-scores of 55–59 indicate mild fatigue, T-scores of 60–70 indicate moderate fatigue, and T-scores &gt;70 indicate severe fatigue. Conclusions The PROMIS Fatigue item bank showed sufficient structural validity, no measurement invariance for demographic characteristics, sufficient cross-cultural validity, and sufficient (internal) reliability in the Dutch general population.

Upper Quadrant Edema Patient-Reported Outcome Measure Is Reliable, Valid, and Efficient for Patients With Lymphatic and Venous Disorders.

The main aims of this study were to (1) create a patient-reported outcome measure (PROM) item bank for measuring the impact of upper quadrant edema (UQE) on physical function by calibrating responses to newly developed items; and (2) assess reliability, validity, and administration efficiency of scores based on computerized adaptive test (CAT) and 10-item short-form (SF) administration modes. This was a retrospective study including data from patients treated in outpatient rehabilitation clinics for UQE that responded to all 27 candidate items at intake. Item response theory model assumptions of unidimensionality, local item independence, item fit, and presence of differential item functioning were evaluated. UQE-CAT- and UQE-SF-generated scores were assessed for reliability, validity, and administration efficiency. The total cohort included 3486 patients (mean [SD] age = 61 [13] years; range = 14-89 years). After removing 2 items, a 25-item solution was supported for its unidimensionality and fit to the item response theory model with reliability estimates of more than 0.93 for scores based on both CAT and SF administration modes. No items demonstrated differential item functioning. Scores discriminated among multiple patient groups in clinically logical ways and were moderately responsive to change with negligible floor and acceptable ceiling effects. CAT scores were generated using an average of 5.6 items (median = 5). Scores on the UQE PROM were reliable, valid, and efficient for assessing perceived physical function of patients with upper quadrant edema; thus, the measure is suitable for research and routine clinical administration. The newly developed UQE PROM is reliable and valid and offers efficient administration modes for assessing perceived physical function of patients with UQE caused by lymphatic and venous disorders, both for research and routine clinical care in busy outpatient rehabilitation settings. As an item response theory-based measure, the UQE PROM allows administration of condition-specific functional questions with low response burden for patients. This study supports a transition to PROMs that are based on modern measurement approaches to achieve high accuracy and efficiency.

Cross‐cultural adaptation and validation of the universal Portuguese‐version of the Pediatric Functional Assessment of Chronic Illness Therapy – Fatigue (pedsFACIT‐F)

ObjectiveTo cross‐culturally adapt and validate the universal Portuguese version of the Pediatric Functional Assessment of Chronic Illness Therapy – Fatigue (pedsFACIT‐F). MethodThe universal Portuguese version of the pedsFACIT‐F was cross‐culturally adapted and validated in 323 children and adolescents aged 8–18 years, 173 healthy individuals, and 150 with chronic diseases (cancer, juvenile idiopathic arthritis, and diabetes). Reliability (internal consistency and test–retest reliability) was assessed. Item response theory model assumptions were evaluated using confirmatory and exploratory factor analyses. Items were calibrated using a graded response model. Differential item functioning was assessed regarding age, gender, and clinical condition (healthy vs. chronic diseases). ResultsNo major cultural adaptations were needed. Internal consistency (Cronbach's alpha=0.84) and test–retest reliability (intraclass correlation coefficient=0.92) were good. CFA (CFI=0.92, TLI=0.90, RMSEA=0.097) and CFE analysis confirmed sufficient unidimensionality. The data also fit the GRM and demonstrated good coverage of the fatigue construct (threshold parameters range: −1.42 to 4.56). No items demonstrated significant differential item functioning. ConclusionThe universal Portuguese version of the pedsFACIT‐F provides a reliable, precise, and valid measure after being assessed by robust psychometric properties. Stability of the measurement properties of the pedsFACIT‐F scale allows its use to assess fatigue in clinical research in Portuguese‐speaking children and adolescents.

Development and calibration of a novel positive mindset item bank to measure health-related quality of life (HRQoL) in Singapore.

BackgroundPositive mindset (PM) is an important domain of health-related quality of life in Singapore, a multi-ethnic urban city state in Southeast Asia. We therefore developed and calibrated a novel item bank to measure and improve PM.MethodsWe developed an initial candidate pool of 48 items from focus groups, in-depth interviews and existing instruments locally developed and validated for use in Singapore. We administered all items in English to a multi-stage sample stratified for age and gender, of subjects with and without medical conditions recruited from the community and a hospital, and calibrated their responses using Samejima’s Graded Response Model. We evaluated a final 36-item bank with respect to Item Response Theory (IRT) model assumptions, model fit, differential item functioning (DIF), concurrent and known-groups validity.ResultsAmong 493 participants (49.3% male, 41.6% above 50 years old, 33% Chinese, Malay and Indian), bifactor model analyses supported unidimensionality: explained common variance of the general factor was 0.86 and omega hierarchical was 0.97. Local independence was deemed acceptable: the average absolute residual correlations were <0.06 and 3.3% of the total item-pair residuals were flagged for local dependence. The overall model fit was adequate and provided good coverage of the PM construct (theta range: -3.6 to +2.4). Five items exhibited DIF with respect to ethnicity and gender, but were retained without modification of scores because they measured important aspects of PM. Scores correlated in the hypothesized direction with a self-reported measure of global health (Spearman’s rho = -0.28, p<0.001) and discriminated between groups of participants with and without a self-reported diagnosis of a mood disorder (p = 0.007) adjusting for age, gender, ethnicity, education and marital status.ConclusionThe 36-item PM item bank demonstrated satisfactory psychometric properties for the English-speaking Singaporean population. IRT model assumptions were sufficiently met and scores showed concurrent and known-groups validity. Future studies to evaluate the validity of PM scores when items are administered adaptively are needed.

Cross-cultural adaptation and validation of the universal Portuguese-version of the Pediatric Functional Assessment of Chronic Illness Therapy – Fatigue (pedsFACIT-F)

ObjectiveTo cross-culturally adapt and validate the universal Portuguese version of the Pediatric Functional Assessment of Chronic Illness Therapy – Fatigue (pedsFACIT-F). MethodThe universal Portuguese version of the pedsFACIT-F was cross-culturally adapted and validated in 323 children and adolescents aged 8–18 years, 173 healthy individuals, and 150 with chronic diseases (cancer, juvenile idiopathic arthritis, and diabetes). Reliability (internal consistency and test–retest reliability) was assessed. Item response theory model assumptions were evaluated using confirmatory and exploratory factor analyses. Items were calibrated using a graded response model. Differential item functioning was assessed regarding age, gender, and clinical condition (healthy vs. chronic diseases). ResultsNo major cultural adaptations were needed. Internal consistency (Cronbach's alpha=0.84) and test–retest reliability (intraclass correlation coefficient=0.92) were good. CFA (CFI=0.92, TLI=0.90, RMSEA=0.097) and CFE analysis confirmed sufficient unidimensionality. The data also fit the GRM and demonstrated good coverage of the fatigue construct (threshold parameters range: −1.42 to 4.56). No items demonstrated significant differential item functioning. ConclusionThe universal Portuguese version of the pedsFACIT-F provides a reliable, precise, and valid measure after being assessed by robust psychometric properties. Stability of the measurement properties of the pedsFACIT-F scale allows its use to assess fatigue in clinical research in Portuguese-speaking children and adolescents.

Adjusting for cross-cultural differences in computer-adaptive tests of quality of life

PurposePrevious studies using the WHOQOL measures have demonstrated that the relationship between individual items and the underlying quality of life (QoL) construct may differ between cultures. If unaccounted for, these differing relationships can lead to measurement bias which, in turn, can undermine the reliability of results.MethodsWe used item response theory (IRT) to assess differential item functioning (DIF) in WHOQOL data from diverse language versions collected in UK, Zimbabwe, Russia, and India (total N = 1332). Data were fitted to the partial credit ‘Rasch’ model. We used four item banks previously derived from the WHOQOL-100 measure, which provided excellent measurement for physical, psychological, social, and environmental quality of life domains (40 items overall). Cross-cultural differential item functioning was assessed using analysis of variance for item residuals and post hoc Tukey tests. Simulated computer-adaptive tests (CATs) were conducted to assess the efficiency and precision of the four items banks.ResultsSplitting item parameters by DIF results in four linked item banks without DIF or other breaches of IRT model assumptions. Simulated CATs were more precise and efficient than longer paper-based alternatives.DiscussionAssessing differential item functioning using item response theory can identify measurement invariance between cultures which, if uncontrolled, may undermine accurate comparisons in computer-adaptive testing assessments of QoL. We demonstrate how compensating for DIF using item anchoring allowed data from all four countries to be compared on a common metric, thus facilitating assessments which were both sensitive to cultural nuance and comparable between countries.

Assessing the Straightforwardly-Worded Brief Fear of Negative Evaluation Scale for Differential Item Functioning Across Gender and Ethnicity.

The Brief Fear of Negative Evaluation Scale (BFNE; Leary Personality and Social Psychology Bulletin, 9, 371-375, 1983) assesses fear and worry about receiving negative evaluation from others. Rodebaugh et al. Psychological Assessment, 16, 169-181, (2004) found that the BFNE is composed of a reverse-worded factor (BFNE-R) and straightforwardly-worded factor (BFNE-S). Further, they found the BFNE-S to have better psychometric properties and provide more information than the BFNE-R. Currently there is a lack of research regarding the measurement invariance of the BFNE-S across gender and ethnicity with respect to item thresholds. The present study uses item response theory (IRT) to test the BFNE-S for differential item functioning (DIF) related to gender and ethnicity (White, Asian, and Black). Six data sets consisting of clinical, community, and undergraduate participants were utilized (N=2,109). The factor structure of the BFNE-S was confirmed using categorical confirmatory factor analysis, IRT model assumptions were tested, and the BFNE-S was evaluated for DIF. Item nine demonstrated significant non-uniform DIF between White and Black participants. No other items showed significant uniform or non-uniform DIF across gender or ethnicity. Results suggest the BFNE-S can be used reliably with men and women and Asian and White participants. More research is needed to understand the implications of using the BFNE-S with Black participants.

Measuring Participation: The Patient-Reported Outcomes Measurement Information System Experience

Bode RK, Hahn EA, DeVellis R, Cella D, on behalf of the Patient-Reported Outcomes Measurement Information System Social Domain Working Group. Measuring participation: the Patient-Reported Outcomes Measurement Information System experience. Objectives To describe the lessons learned in the initial development of Patient-Reported Outcomes Measurement Information System social function item banks. Design Development and testing of 2 item pools within a general population to create item banks that measure ability to participate and satisfaction with participation in social activities. Setting Administration via the Internet. Participants General population members (N=956) of a national polling organization registry participated; data for 768 and 778 participants were used in the analysis. Interventions Not applicable. Main Outcome Measures Measures of ability to participate and satisfaction with participation in social activities. Results Fifty-six items measuring the ability to participate were essentially unidimensional but did not fit an item response theory model. As a result, item banks were not developed for these items. Of the 56 items measuring satisfaction with participation, 14 items measuring social roles and 12 items measuring discretionary activities were unidimensional and met item response theory model assumptions. Two 7-item short forms were also developed. Conclusions Four lessons, mostly concerning item content, were learned in the development of banks measuring social function. These lessons led to item revisions that are being tested in subsequent studies.

Better assessment of physical function: item improvement is neglected but essential

IntroductionPhysical function is a key component of patient-reported outcome (PRO) assessment in rheumatology. Modern psychometric methods, such as Item Response Theory (IRT) and Computerized Adaptive Testing, can materially improve measurement precision at the item level. We present the qualitative and quantitative item-evaluation process for developing the Patient Reported Outcomes Measurement Information System (PROMIS) Physical Function item bank.MethodsThe process was stepwise: we searched extensively to identify extant Physical Function items and then classified and selectively reduced the item pool. We evaluated retained items for content, clarity, relevance and comprehension, reading level, and translation ease by experts and patient surveys, focus groups, and cognitive interviews. We then assessed items by using classic test theory and IRT, used confirmatory factor analyses to estimate item parameters, and graded response modeling for parameter estimation. We retained the 20 Legacy (original) Health Assessment Questionnaire Disability Index (HAQ-DI) and the 10 SF-36's PF-10 items for comparison. Subjects were from rheumatoid arthritis, osteoarthritis, and healthy aging cohorts (n = 1,100) and a national Internet sample of 21,133 subjects.ResultsWe identified 1,860 items. After qualitative and quantitative evaluation, 124 newly developed PROMIS items composed the PROMIS item bank, which included revised Legacy items with good fit that met IRT model assumptions. Results showed that the clearest and best-understood items were simple, in the present tense, and straightforward. Basic tasks (like dressing) were more relevant and important versus complex ones (like dancing). Revised HAQ-DI and PF-10 items with five response options had higher item-information content than did comparable original Legacy items with fewer response options. IRT analyses showed that the Physical Function domain satisfied general criteria for unidimensionality with one-, two-, three-, and four-factor models having comparable model fits. Correlations between factors in the test data sets were > 0.90.ConclusionsItem improvement must underlie attempts to improve outcome assessment. The clear, personally important and relevant, ability-framed items in the PROMIS Physical Function item bank perform well in PRO assessment. They will benefit from further study and application in a wider variety of rheumatic diseases in diverse clinical groups, including those at the extremes of physical functioning, and in different administration modes.

Practical Issues in the Application of Item Response Theory

Item response theory (IRT) is increasingly being applied to health-related quality of life instrument development and refinement. This article discusses results obtained using categorical confirmatory factor analysis (CCFA) to check IRT model assumptions and the application of IRT in item analysis and scale evaluation. To demonstrate the value of CCFA and IRT in examining a health-related quality of life measure in children and adolescents. This illustration uses data from 10,241 children and their parents on items from the 4 subscales of the PedsQL 4.0 Generic Core Scales. CCFA was applied to confirm domain dimensionality and identify possible locally dependent items. IRT was used to assess the strength of the relationship between the items and the constructs of interest and the information available across the latent construct. CCFA showed generally strong support for 1-factor models for each domain; however, several items exhibited evidence of local dependence. IRT revealed that the items generally exhibit favorable characteristics and are related to the same construct within a given domain. We discuss the lessons that can be learned by comparing alternate forms of the same scale, and we assess the potential impact of local dependence on the item parameter estimates. This article describes CCFA methods for checking IRT model assumptions and provides suggestions for using these methods in practice. It offers insight into ways information gained through IRT can be applied to evaluate items and aid in scale construction.

Psychometric Evaluation and Calibration of Health-Related Quality of Life Item Banks

The construction and evaluation of item banks to measure unidimensional constructs of health-related quality of life (HRQOL) is a fundamental objective of the Patient-Reported Outcomes Measurement Information System (PROMIS) project. Item banks will be used as the foundation for developing short-form instruments and enabling computerized adaptive testing. The PROMIS Steering Committee selected 5 HRQOL domains for initial focus: physical functioning, fatigue, pain, emotional distress, and social role participation. This report provides an overview of the methods used in the PROMIS item analyses and proposed calibration of item banks. Analyses include evaluation of data quality (eg, logic and range checking, spread of response distribution within an item), descriptive statistics (eg, frequencies, means), item response theory model assumptions (unidimensionality, local independence, monotonicity), model fit, differential item functioning, and item calibration for banking. Summarized are key analytic issues; recommendations are provided for future evaluations of item banks in HRQOL assessment.

Using item response theory in test score equating

In this chapter, the theoretical advantages that have been offered for using item response theory (IRT) in the test equating process are discussed. In addition, IRT equating research is reviewed with regard to certain important equating issues addressed in this research. These issues fall into four general categories: (1) population invariance of equating results and sample selection in equating, (2) properties of linking items used in anchor test equatings, (3) further examinations of the use of IRT in the vertical equating of tests, and (4) robustness of IRT equating to violations of underlying IRT model assumptions. The chapter concludes with a number of practical suggestions to guide researchers interested in IRT for test equating purposes.

An Application of Item Response Theory to Certifying Examinations in Internal Medicine

This study assessed the suitability of item response theory (IRT) for medical examination data. The specific purposes were (I) to see whether the American Board of Internal Medicine (ABIM) Certifying Examination data met IRT model assumptions and (2) to apply the one-parameter and three-parameter IRT models to the data and observe whether the expected benefits were obtained Analysis of examinees' responses to single-best-answer items supported the general assumptions of local independence, unidimensionality, and nonspeededness. The specific assumptions of the three-parameter model were met, in that items differed in discrimination and guessing. The estimated ability and item parameters were not initially as Tables as hoped, because of a few poorly estimated parameters and possibly the homogeneity of the examinee group and consequently the large number of items with poor discrimination. Future work needs to include content experts to help understand why some items do not fit and to ensure that the retained items result in a content-valid examination.